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Foreword 

The  Federal  Information  Processing  Standards  Publication  Series  of  the  National 
Bureau  of  Standards  is  the  official  publication  relating  to  standards  adopted  and 
promulgated  under  the  provisions  of  Public  Law  89-306  (Brooks  Act)  and  under 
Part  6  of  Title  15,  Code  of  Federal  Regulations.  Under  P.L.  89-306  the  Secretary  of 
Commerce  has  important  responsibilities  for  improving  the  utilization  and  effectiveness 
of  computer  systems  in  the  Federal  Government.  To  carry  out  the  Secretary’s 
responsibilities,  the  NBS,  through  its  Institute  for  Computer  Sciences  and  Technology, 
provides  leadership,  technical  guidance,  and  coordination  of  Government  efforts  in  the 
development  of  technical  guidelines  and  standards  in  these  areas. 

The  complexity  of  managing  today’s  ADP  computer  facility  is  compounded  by  the 
growing  technological  complexity  and  interaction  of  the  resources  being  managed. 
This  technological  complexity  demands  that  highly  specialized  tools  and  techniques 
be  available  to  the  ADP  manager  so  that  he  may  more  effectively  and  efficiently 
manage  his  installation.  The  objective  of  a  Computer  Performance  Management 
program  is  the  application  of  this  contemporary,  specialized  technology  in  support  of 
good  management.  This  document  introduces  the  Federal  ADP  manager  to  Computer 
Performance  Management  and  recommends  the  establishment  of  such  a  program  at  all 
Federal  ADP  facilities. 


Ruth  M.  Davis,  Director 
Institute  for  Computer  Sciences 
and  Technology 


Abstract 

A  Computer  Performance  Management  (CPM)  program  is  any  structured  effort 
to  measure  and  evaluate  the  performance  of  installed  computer  systems  in  support  of 
established  management  goals  and  objectives.  The  purpose  of  this  publication  is  to  in¬ 
troduce  the  Federal  ADP  manager  to  the  subject  of  CPM,  to  provide  general  assistance 
to  Federal  ADP  managers  in  planning  and  organizing  a  CPM  program,  and  to  recom¬ 
mend  the  establishment  of  CPM  programs  at  all  Federal  ADP  facilities.  Guidance  is 
presented  on  the  use  of  performance  measures  in  four  major  areas  of  management 
responsibility.  The  role  of  the  ADP  manager  and  the  expected  resources  required  in 
instituting  a  CPM  program  are  discussed. 

Key  words:  Computer  performance  evaluation;  computer  performance  management: 
Federal  Information  Processing  Standards;  performance  measures. 
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Applicability.  This  guideline  is  intended  as  a  reference  document  of  recommended  practices  for 
general  use  throughout  the  Federal  Government  in  planning  and  organizing  Computer  Perform¬ 
ance  Management  programs. 

Implementation.  Federal  departments  and  agencies  are  responsible  for  issuing  instructions  for  the 
use  of  this  guideline  by  their  operating  units  and  for  assuring  that  it  is  distributed  or  made  avail¬ 
able  to  appropriate  organizations  and  individuals  concerned.  All  comments  and  critiques  are  wel¬ 
come  and  will  be  considered  in  future  revisions.  These  should  be  addressed  to  the  Systems  and 
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This  document  is  intended  to  serve  as  an  introduction  to  Computer  Performance  Management. 
Because  agencies  have  unique  performance  goals,  no  specific  guidance  can  be  prescribed  as  to 
recommended  levels  of  “good”  performance.  As  new  insight  into  Computer  Performance  Manage¬ 
ment  is  gained,  this  guideline  will  be  modified  accordingly. 
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I.  Introduction 

The  purpose  of  this  Guideline  is  to  introduce 
Federal  ADP  management  to  the  concept  of 
Computer  Performance  Management  (CPM), 
and  to  recommend  the  establishment  of  a  CPM 
program  at  all  Federal  ADP  facilities.  For  pur¬ 
poses  of  this  Guideline,  an  “ADP  facility”  will 
be  understood  to  mean  the  complete  set  of 
resources  dedicated  to  meeting  an  agency’s  re¬ 
quirements  for  automated  information  process¬ 
ing.  “Management”  of  such  a  facility  may  either 
converge  at  a  single  point  in  the  organizational 
hierarchy,  or  it  may  be  divided  among  several 
collateral  points  of  functional  responsibility. _  A 
“CPM  program”  is  any  structured  effort,  in- 
house  or  otherwise,  to  measure  and  evaluate 
the  performance  of  a  computer  facility  in  sup¬ 
port  of  established  management  goals  and 
objectives. 

The  complexity  of  modern  computer  systems 
requires  that  the  ADP  facility  manager  have  at 
hand  a  wide  range  of  information  to  fulfill  his 
responsibilities  effectively.  These  responsibili¬ 
ties  range  from  satisfying  the  requirements  of 
his  users  to  notifying  upper  management  of 
problems  and  status.  Each  area  of  responsibility 
calls  for  its  own  set  of  specific,  quantitative  and 
often  technical  information.  The  application  of 
measurement  techniques  and  evaluation  meth¬ 
odologies  to  obtain  this  information  is  central 
to  any  CPM  program. 

This  Guideline  begins  with  an  explanation  of 
the  relationship  between  CPM  and  the  func¬ 
tional  responsibilities  of  the  ADP  facility  mana¬ 
ger.  Specific  responsibilities  to  be  considered 
include:  (1)  service  to  users;  (2)  management 
of  resources;  (3)  communication  with  upper 
management ;  and  (4)  vendor  relations.  A  num¬ 
ber  of  recommendations  are  then  made  to  assist 
the  ADP  manager  in  initiating  a  CPM  program 
at  his  own  facility. 


II.  CPM  and  Management 
Responsibilities 

The  problems  and  responsibilities  that  the 
ADP  manager  must  face  are  similar  to  those 
encountered  by  managers  in  other  functional 
areas.  The  goal  of  maximizing  productivity  at 
minimum  cost  applies  to  the  ADP  manager  as 
well  as  to  the  manager  of  a  manufacturing 
plant.  His  need  for  timely  and  accurate  informa¬ 
tion  with  which  to  plan  for  future  expansion  is 
as  acute  as  the  needs  of  the  inventory-control 
manager  of  a  large  department  store.  But  while 
the  growth  of  computer  technology  has  helped 
to  ease  the  burden  and  increase  the  effectiveness 
of  managers  in  nearly  every  other  aspect  of 
agency  operations,  it  has  turned  the  ADP 


facility  manager’s  job  into  a  sometimes  over¬ 
whelming  challenge.  He  may  be  charged  with 
the  responsibility  for  managing  a  multi-million 
dollar  resource  that  operates  at  electronic  speed 
on  hundreds  of  independent  problems  at  a  time. 
Today’s  largest  computers  can  process  thous¬ 
ands  of  different  jobs  from  as  many  different 
sources  each  day.  Except  for  sporadic  messages 
on  the  operator’s  console,  and  the  visible  move¬ 
ment  of  tapes,  disks,  and  printers,  most  of 
what  determines  how  well  a  computer  system 
does  its  work  is  concealed  within  the  circuitry 
of  the  hardware  itself.  It  cannot  be  observed, 
measured  or  evaluated  without  the  application 
of  specialized  technology.  Yet  many  of  the  im¬ 
portant  decisions  that  a  manager  must  make 
about  the  ADP  resource  depend  upon  a  detailed 
and  precise  understanding  of  these  invisible 
events  and  what  they  signify. 

Consider  a  few  of  the  decisions  that  the  ADP 
facility  manager  faces  nearly  every  day  of  his 
working  life.  Are  user  complaints  about  poor 
service  justified?  If  so,  what  is  the  most 
economical  and  feasible  way  to  remove  or  mini¬ 
mize  the  cause?  If  not,  what  kind  of  human 
factors  may  have  led  to  user  dissatisfaction 
and  how  should  the  facts  about  service  levels  be 
communicated  to  them?  What  kinds  of  econo¬ 
mies  or  broad  service  improvements  could  be 
realized  by  minor  modifications  to  user  require¬ 
ments?  How  are  user  requirements  expected  to 
change  in  the  next  one  to  five  years,  and  with 
what  effect  on  present  resources?  As  for  the 
facility  itself,  how  many  operator  shifts  are 
needed  to  handle  the  existing  workload?  How 
should  the  computer  room  be  laid  out  to  opti¬ 
mize  operator  efficiency?  What  kind  of  back¬ 
ground  should  be  required  of  support  personnel, 
and  what  kind  of  training  should  they  receive 
after  they  arrive?  Is  the  amount  of  downtime 
the  system  suffers  reasonable,  given  the  state 
of  contemporary  hardware  electronics  and  soft¬ 
ware  engineering?  Is  the  apparent  slowdown 
in  throughput  at  peak  load  hours  acceptable  to 
users?  To  management?  Is  there  anything  that 
can  be  done  with  the  present  configuration  to 
enhance  its  performance  at  peak  hours?  If  so, 
will  the  benefits  justify  the  cost  of  the  enhance¬ 
ment?  If  not,  where  is  the  bottleneck  in  the 
system  and  what  is  the  most  cost-effective  way 
to  alleviate  it? 

Questions  like  these  could  fill  many  pages. 
The  point  is  that  having  reasoned,  quantitative, 
well-documented  answers  to  these  questions 
largely  determines  the  effectiveness  with  which 
a  manager  manages  an  ADP  facility,  instead  of 
merely  supervising  it.  Sometimes  the  answers 
may  provide  the  Federal  Government  with  sub¬ 
stantial  dollar  savings,  or  significantly  enhance 
an  agency’s  ability  to  carry  out  its  mission.  The 
more  immediate  objective  of  CPM,  however,  is 
to  regain  control  of  a  complex,  costly  and  criti- 


5 


FIPS  PUB  49 


cal  resource  through  a  quantitative  understand¬ 
ing  of  how  that  resource  performs  and  of  the 
alternatives  that  are  available  to  make  it  per¬ 
form  more  effectively  and  efficiently.  In  particu¬ 
lar,  the  ADP  manager  must  be  effective  and 
therefore  well-informed  in  at  least  four  major 
areas:  (1)  user  requirements ;  (2)  management 
of  and  planning  for  ADP  resources ;  (3)  com¬ 
municating  with  upper  management ;  and  (4) 
vendor  relations. 

A.  User  Requirements 

ADP  management’s  greatest  responsibility 
is  to  its  users.  They,  after  all,  are  the  reason 
the  ADP  facility  exists.  Although  specific  user 
needs  are  often  not  clearly  stated,  several 
categories  of  user  requirements  do  appear  to  be 
applicable  across  vendor  lines  and  across  ADP 
installations. 

1.  User  Service 

(a)  Timeliness 

Most  ADP  users  have  deadlines  to  meet, 
and  rely  on  the  computer  to  meet  them.  This 
is  as  true  during  the  project  development  phase 
as  in  the  production  phase,  and  is  often  reflected 
in  the  user’s  need  for  rapid  turnaround  time. 
In  order  to  fulfill  his  own  responsibilities,  the 
ADP  manager  must  know  first  what  these  turn¬ 
around  time  requirements  are,  and,  second, 
whether  his  center  is  meeting  them.  Surveying 
users  may  not  be  a  practical  approach  to  defin¬ 
ing  such  requirements,  and  waiting  for  users  to 
pound  on  the  ADP  manager’s  door  is  no  solution 
to  monitoring  levels  of  service.  A  better  ap¬ 
proach  would  be  to  obtain  detailed  information 
for  each  job  class  (e.g.,  the  number  of  jobs  sub¬ 
mitted,  the  percent  of  jobs  meeting  published 
turnaround  times)  from  system  accounting  log 
files.  This  information  is  invaluable  for  resolv¬ 
ing  questions  about  batch  job  turnaround  time, 
at  least  for  the  job’s  life  in  the  system.  If  a 
critical  delay  is  suspected  outside  the  system,  a 
time-stamping  procedure  may  be  instituted  to 
record  actual  batch  job  submission  and  pick-up 
times,  or  other  manual  operations  such  as  read¬ 
ing  in  cards,  bursting  output,  etc. 

Although  degradation  of  turnaround  time  by 
a  few  seconds  is  not  noticeable  to  the  batch  user, 
an  equivalent  delay  in  response  time  can  be 
very  irritating  to  the  interactive  user.  The  de¬ 
velopment  of  remote  terminal  emulators 
(RTE’s)  and  of  intelligent  hardware  monitors 
has  made  it  possible  to  accurately  measure  in¬ 
teractive  response  time  at  a  large  number  of 
terminals.  From  the  manager’s  point  of  view, 
RTE-like  monitors  can  provide  him  with  an 
accurate  record  of  user  interactions,  which  tell 
him  and  his  interactive  users  exactly  what  level 
of  service  they  are  actually  receiving. 


(b)  Accessibility 

The  location  and  quantity  of  remote  batch 
and  interactive  terminals  can  do  much  to  affect 
a  user’s  attitude  toward  an  ADP  center.  If  the 
user  has  difficulty  in  finding  an  unused  terminal, 
he  will  have  little  regard  for  the  ADP  mana¬ 
ger’s  desire  to  satisfy  his  immediate  require¬ 
ments.  Measurement  techniques  can  be  used  to 
record  port  numbers  and  sign-on  ID’s  of  every 
remote  access  to  the  center.  This  data  can  then 
be  analyzed  to  provide  a  clear  picture  of  termi¬ 
nal  demand  which  may  suggest  a  more  balanced 
placement  of  available  terminals. 

(c)  Reliability 

A  brief  system  crash  may  have  little  im¬ 
pact  on  the  total  turnaround  time  of  a  batch 
user’s  job,  but  it  may  be  unnerving  to  an  inter¬ 
active  user  sitting  at  a  terminal  wondering 
when  the  system  will  come  up  again  and  how 
much  of  his  work  may  have  been  lost  in  the 
process.  Without  sufficient  information  as  to 
the  frequency  and  duration  of  system  crashes, 
the  ADP  manager  has  no  real  feel  for  the 
quality  of  service  he  is  providing  his  users. 
Simple  measures  such  as  mean-time-between- 
failures  (MTBF)  and  mean-time-to-repair 
(MTTR)  can  provide  this  information. 

(d)  Availability 

System  availability  is  usually  defined  as 
the  percent  of  scheduled  time  during  which  the 
computer  system  is  operational.  There  is  an¬ 
other  aspect  of  availability,  however,  which 
concerns  the  nature  of  scheduled  time  itself. 
For  example,  heavy  user  demand  during  week¬ 
end  or  after-hours  periods  may  necessitate  the 
assignment  of  additional  personnel  or  even  the 
establishment  of  an  additional  shift.  Informa¬ 
tion  concerning  the  magnitude  and  type  of 
demands  across  several  weeks  can  be  used  to 
determine  the  appropriate  hours  of  scheduled 
time  for  each  major  subsystem. 

2.  User  Support 

Support  functions  such  as  training  seminars, 
user  consultation,  manuals,  and  regular  center 
news  bulletins  are  perhaps  the  most  elusive 
user  requirements  to  determine  and  evaluate. 
However,  analysis  of  the  type  and  frequency 
of  user  errors,  use  of  existing  vendor  products, 
and  user  inquiries  to  support  personnel  usually 
provides  the  ADP  manager  with  enough  in¬ 
formation  to  make  intelligent  and  timely  train¬ 
ing  and  staffing  decisions.  In  addition,  the  dili¬ 
gent  logging  and  analysis  of  user-reported  prob¬ 
lems  and  complaints  may  indicate  possible 
system-wide  hardware  or  software  problems. 
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3.  User  Chargeback 

Many  facility  managers  have  found  that  the 
key  to  quantifying  and  controlling  user  require¬ 
ments  is  the  introduction  of  a  system  for 
“charging”  users  on  the  basis  of  equipment  or 
services  they  require.  The  “charges”  may  only 
be  for  accounting  purposes  and  not  involve  an 
actual  transfer  of  funds.  A  chargeback  system 
must  meet  at  least  two  minimal  design  criteria: 
(1)  charges  should  have  some  clear-cut  relation¬ 
ship  to  actual  resource  utilization  which  users 
can  easily  understand ;  (2)  charges  should  be 
approximately  repeatable  for  a  given  job 
operating  on  the  same  data  on  a  given  system. 
Once  accountability  for  computer  resource  utili¬ 
zation  is  established,  users  are  frequently  en¬ 
couraged  to  cut  down  on  wasteful  uses  of  com¬ 
puter  time.  Others  may  restructure  their 
legitimate  workload  to  minimize  “costs,”  and  if 
the  charging  formula  has  been  carefully  de¬ 
signed,  the  effect  will  be  a  reduction  in  user 
demand  on  critical  system  resources.  In  addition 
to  modifying  user  behavior,  a  chargeback 
scheme  facilitates  user  planning  and  budgeting 
for  future  computer  service  needs.  This,  in  turn, 
helps  the  facility  manager  to  forecast  overall 
workload  growth  patterns  and  to  anticipate  the 
need  for  new  equipment,  space,  personnel,  or 
procedures. 

Although  resource  utilization  information 
on  which  to  base  user  charges  is  often  provided 
by  vendor-supplied  accounting  logs,  it  may  be 
necessary  to  supplement  this  data  with  “home¬ 
brew”  modifications  or  additional  system  instru¬ 
mentation.  Such  modifications  are  frequently 
motivated  by  a  need  for  more  accurate  and 
repeatable  job  charges,  or  by  the  desire  to  have 
actual  dollar  costs  printed  at  the  end  of  each 
job.  A  particularly  effective  example  of  this 
latter  service  is  the  projected  job  charge  that 
informs  the  user  of  what  his  charges  would 
have  been  had  he  allocated  only  as  many  system 
resources  as  his  job  actually  used. 

B.  Resource  Management 

The  most  obvious  responsibility  of  the  ADP 
facility  manager  is  the  direct  control  of  re¬ 
sources  (equipment,  space,  personnel,  etc.) 
placed  under  his  authority.  He  must  continually 
balance  his  resource  costs  against  the  dynamic 
requirements  of  his  users.  Having  established 
a  charging  algorithm  upon  which  job  billings 
are  based,  for  example,  the  ADP  manager  may 
find  over  time  that  his  costs  increase  to  the 
point  that  they  exceed  revenues.  This  may  be 
due  to  increased  personnel  costs,  decreased  de¬ 
mand,  inflationary  increases  in  the  cost  of  equip¬ 
ment  and  supplies,  or  procurement  of  additional 
hardware  or  software. 

For  facilities  with  chargeback  systems,  one 
solution  to  increased  costs  is  relatively  straight¬ 


forward  :  raise  the  price  of  ADP  services.  A 
more  acceptable  solution  is  to  reduce  costs  of 
processing  a  given  workload  through  the  appli¬ 
cation  of  measurement  techniques  and  the  use 
of  fairly  simple  time-motion  studies.  Elimina¬ 
tion  of  underutilized  peripheral  equipment, 
reduction  in  overtime,  and  elimination  of  un¬ 
needed  operator  shifts  are  examples  of  direct 
cost  savings  resulting  from  computer  perform¬ 
ance  evaluation  efforts. 

Resource  management  also  involves  planning 
for  the  future.  This  implies  that  the  ADP  mana¬ 
ger  must  have  detailed  information  concerning 
the  current  workload :  its  history  of  growth,  its 
present  resource  demands,  its  likely  growth  in 
the  future.  Number  of  jobs  completed  per 
month,  percent  utilization  of  major  system  re¬ 
sources,  hours  of  system  availability  are  sev¬ 
eral  measures  applicable  to  this  problem.  Per¬ 
formance  data  is  thus  valuable  not  only  for 
enhancing  the  present  system,  but  also  for 
constructing  models  of  future  resource  require¬ 
ments. 

C.  Communicating  With  Upper  Management 

The  ADP  manager  has  a  responsibility  to 
report  to  his  upper  management  on  the  status, 
performance  and  requirements  of  his  facility. 
His  reports  should  include,  at  a  minimum,  sum¬ 
mary  information  concerning  the  previously 
discussed  areas  of  responsibility.  Although  the 
form  of  such  reports  is  a  matter  of  taste  and  a 
function  of  the  reporting  requirements  of  upper 
management,  some  general  guidance  can  be 
given : 

1.  Status  reports  should  be  regular,  concise, 
and  preferably  graphical  in  nature. 

2.  The  amount  of  information  reported  should 
not  exceed  management  requirements. 
“Too  much,  too  often”  is  a  problem  com¬ 
mon  to  many  performance  reporting 
schemes. 

3.  Information  should  be  at  a  level  of  ab¬ 
straction  which  upper  management  can 
easily  digest  and  understand,  but  sufficient 
to  support  the  decision-making  process. 

4.  The  reports  should  compare  the  center’s 
current  level  of  performance  against  a  set 
of  predefined  performance  goals. 

Performance  measures  are  thus  not  only  a  basis 
for  satisfying  the  informational  needs  of  the 
ADP  manager,  but  also  an  effective  means  of 
communication  between  different  levels  of  man¬ 
agement  responsibility  within  the  organization. 

D.  Vendor  Relations 

Most  ADP  managers  interface  regularly  with 
vendor  marketing  and  technical  support  person¬ 
nel.  The  relationship  with  technical  personnel 
becomes  especially  important  in  the  areas  of 
hardware  and  software  reliability  and  mainte- 
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nance.  A  joint  review  of  hardware  and  software 
performance  data  on  a  regular  basis  by  ADP 
personnel  and  vendor  representatives  will  help 
foster  mutual  respect  and  understanding.  Track¬ 
ing  the  frequency  of  tape  and  disk  parity  errors, 
the  cause  and  duration  of  system  crashes,  and 
variations  in  other  system  performance  meas¬ 
ures  will  do  much  to  raise  the  facility  manager’s 
concerns  from  the  level  of  intuition  to  a  demon¬ 
stration  of  hard  fact  with  which  the  vendor 
must  deal.  The  ability  to  identify  the  source 
as  well  as  the  existence  of  errors  becomes 
especially  important  in  a  multi-vendor  ADP 
environment. 

Proposed  hardware  and  software  modifica¬ 
tions  must  be  evaluated  to  project:  (1)  utility; 
(2)  cost  effectiveness ;  and  (3)  impact  on  total 
system  performance.  Major  on-site  modifica¬ 
tions  to  a  card  punch  in  order  to  remedy  an 
off-punch  problem  may  require  the  unit  to  be 
down  for  a  significant  period  of  time.  Hence,  a 
complete  replacement  might  be  more  desirable. 
A  modification  to  the  system  scheduler  which 
is  intended  to  increase  batch  throughput  but 
which  inadvertently  degrades  interactive  re¬ 
sponse  time  fails  to  consider  total  system  per¬ 
formance.  Measurement  tools  and  techniques 
can  be  used  not  only  to  detect  performance 
problems,  but  also  to  anticipate  them  and  to 
prevent  their  occurrence. 

III.  Instituting  the  CPM  Program 

Although  measurement  and  evaluation  tech¬ 
niques  are  available  to  support  the  efficient  and 
effective  management  of  an  ADP  center,  the 
question  facing  today’s  ADP  manager  is  how 
to  introduce  this  new  technology  into  his  own 
facility.  How  often  should  the  information 
obtained  from  performance  data  be  reported  to 
the  ADP  manager,  for  example?  In  what  form 
should  it  be  reported  ?  What  is  the  ADP  mana¬ 
ger’s  role  in  instituting  CPM  procedures?  The 
following  paragraphs  present  a  number  of  such 
issues  that  should  be  considered  in  inaugurating 
a  performance  management  program. 

A.  CPM  Reporting 

Figure  1  depicts  the  life-cycle  of  a  typical 
computer  system,  progressing  from  an  analysis 
of  requirements  to  the  final  installation,  opera¬ 
tion,  and  enhancement  of  the  selected  system. 
In  each  phase  of  the  computer  life-cycle,  meas¬ 
urement  and  evaluation  play  a  major  role  in 
satisfying  the  informational  needs  of  the  ADP 
manager.  As  noted  earlier,  performance  data  is 
as  useful  during  the  requirements  analysis 
phase  as  it  is  during  the  system  enhancement 
phase.  Every  installation,  regardless  of  size, 
should  have  some  form  of  reporting  during  each 
phase  of  its  system’s  life-cycle. 


The  types  of  data  to  be  collected  and  reported 
should  be  determined  not  by  their  mere  avail¬ 
ability,  but  by  the  informational  requirements 
of  the  ADP  manager.  These  informational  re¬ 
quirements  are  in  turn  determined  by  the  ADP 
manager’s  scope  of  responsibility.  Each  report 
should  provide  a  historical  trend  of  the  center’s 
performance  which  is  updated  on  a  regular  basis 
(depending  upon  the  nature  and  importance  of 
the  information),  and  should  contain  specified 
performance  criteria.  These  criteria  may  be 
translated  into  control  limits  on  the  perform¬ 
ance  charts.  A  control  limit  is  a  value  chosen 
to  represent  the  boundaries  of  acceptable  per¬ 
formance  for  a  given  system  variable.  Some  of 
these  variables  and  their  associated  control 
limits  may  be  “objective-directed” — that  is, 
they  indicate  the  center’s  ability  to  meet  cer¬ 
tain  specified  objectives  (e.g.,  one-hour  average 
batch  turnaround  time).  Others  are  “process- 
directed,”  indicating  the  level  of  performance 
of  internal  system  resources  (e.g.,  the  CPU, 
disks,  memory).  When  control  limits  are  ex¬ 
ceeded,  an  exception  report  is  generated  and, 
when  appropriate,  an  in-depth  study  may  be 
recommended  to  determine  the  specific  cause  (s) 
for  the  exception  and  appropriate  remedies  for 
its  correction.  Sample  reports  consistent  with 
the  ADP  manager’s  responsibility  to  his  users 
in  the  areas  of  turnaround  time,  reliability,  and 
user  support  (see  Section  II. A)  appear  in  the 
Appendix. 

Determining  control  limit  values  is  highly 
dependent  on  the  constraints  and  resources  of 
the  ADP  center,  and  indeed  on  the  goals  of  the 
organization.  Frequently,  past  performance  has 
been  used  as  a  standard  against  which  current 
performance  is  evaluated.  Although  past  per¬ 
formance  does  not  necessarily  mean  good  per¬ 
formance,  it  is  a  reliable  indicator  of  the  base¬ 
line  system’s  natural  reaction  to  various 
workload  demands. 

Finally,  the  performance  reports  generated  by 
an  ADP  center  should  always  remain  highly 
visible,  especially  to  its  user  community.  Publi¬ 
cation  of  “performance  charts”  in  the  center’s 
regular  newsletter  is  an  excellent  vehicle  for 
accomplishing  this. 

B.  Management’s  Role 

The  ADP  facility  manager  should  play  a  cen¬ 
tral  role  in  instituting  and  overseeing  a  CPM 
program.  The  scope  and  objectives  of  the  pro¬ 
gram  should  be  clearly  established  from  the 
beginning  so  as  to  preclude  any  misconceptions 
and  unnecessary  data  collection.  Perhaps  most 
importantly,  subsequent  control  and  review  by 
the  ADP  manager  is  needed  to  guarantee  that 
the  program  is  continually  meeting  the  organi¬ 
zation’s  informational  needs,  in  addition  to 
insuring  that  the  morale  of  the  program  per¬ 
sonnel  is  maintained. 
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1.  Define  Scope  and  Objectives 

Prior  to  instituting  any  CPM  reporting  pro¬ 
cedures,  the  ADP  manager  should  first  have  a 
clear  understanding  and  definition  of  the  scope 
of  his  responsibilities.  Next,  the  objectives  of 
the  proposed  CPM  program  should  be  estab¬ 
lished  and  the  information  requirements  should 
be  defined  in  order  to  insure  that  only  pertinent 
data  will  be  collected  during  the  later  phases  of 
the  program.  Finally,  the  nature  of  the  actual 
reports  to  be  produced,  the  frequency  with 
which  they  should  be  produced,  and  the  per¬ 
formance  criteria  related  to  each  should  be 
determined.  Once  again,  the  reports  should 
reflect  enough,  and  only  enough,  information  for 
the  recipients  to  function  effectively  within  the 
scope  of  their  responsibility. 

2.  Determine  Approach 

A  successful  CPM  program  requires  skilled 
personnel  who  are  intimately  familiar  with  the 
computer  resources  being  measured  and  the 
tools  being  used,  and  who  have  the  ability  to 
properly  analyze  and  interpret  the  measure¬ 
ment  data.  Two  possible  sources  thus  exist  for 
establishing  CPM  reporting  procedures:  in- 
house  personnel  and  outside  consultants. 

For  installations  with  qualified  and  available 
personnel,  the  in-house  approach  is  preferred. 
This  not  only  allows  for  better  project  control, 
but  has  the  additional  advantage  that  in-house 
personnel  are  more  aware  of  the  objectives  and 
internal  workings  of  the  organization.  When 
qualified  in-house  personnel  are  not  available, 
an  outside  consultant  should  be  used.  Some  ad¬ 
vantages  of  the  outside-consultant  approach 
are  its  cost-effectiveness  (for  organizations 
with  no  available  performance  measurement 
resources — i.e.,  personnel,  monitors,  etc.),  its 
objectivity,  and  the  opportunity  for  knowledge- 
transfer  from  the  consultants  to  in-house 
contacts.  Possible  disadvantages  include  the 
consultants’  lack  of  familiarity  with  the  internal 
operations  of  the  organization,  and  the  possible 
friction  that  may  arise  between  the  consultants 
and  in-house  personnel. 

3.  Control  and  Review 

The  failure  of  many  performance  improve¬ 
ment  efforts  has  been  traced  to  the  lack  of 
genuine  management  interest  and  support.  Too 
often  the  ADP  manager  supports  a  performance 
measurement  group  during  its  initial  phases, 
but  then  gradually  loses  interest.  This  decline 
in  interest  is  usually  a  result  of  false  hopes  and 
unrealistic  expectations  for  cost  savings. 
Recognition  of  CPM  as  a  valuable  source  of 
information  and  a  necessary  adjunct  of  ADP 
management,  as  well  as  a  technology  for  reduc¬ 


ing  costs  and  increasing  efficiency,  will  do  much 
to  sustain  an  ADP  manager’s  active  interest  and 
support. 

Finally,  the  total  CPM  program  should  under¬ 
go  a  periodic  review.  Changes  in  informational 
needs  should  be  reflected  in  new  CPM  reports. 
Existing  reports  should  be  examined  to  deter¬ 
mine  their  current  relevancy.  Too  often  reports 
continue  to  be  generated  when  the  need  for 
them  has  long  since  passed. 

C.  Resource  Requirements 

Perhaps  the  most  significant  costs  of  any 
CPM  program  are  those  incurred  during  start¬ 
up.  Such  costs  usually  include:  the  program’s 
initial  planning,  system  modifications  to  acquire 
the  necessary  data,  acquisition  of  commercial 
products  and  packages,  and  the  development  of 
report  generation  mechanisms.  Once  underway, 
continuous  reporting  demands  other  costs  and 
resources:  system  overhead  to  collect  perform¬ 
ance  data,  processing  time  to  analyze  the  data, 
and  manpower  to  interpret  the  data  and  main¬ 
tain  the  reporting  system.  In  addition,  in-depth 
studies  prompted  by  exception  reports  may  re¬ 
quire  the  use  of  rather  sophisticated  tools — 
notably,  hardware  and  software  monitors.  The 
cost  of  these  tools  varies  with  the  type  and 
accuracy  of  data  to  be  obtained.  The  additional 
costs  of  personnel  and  training  may,  once  again, 
make  it  more  cost-effective  to  hire  outside  con¬ 
sultants  for  such  one-time  studies. 

IV.  Recommendation 

It  is  the  conclusion  of  FIPS  Task  Group  10 
that  only  through  the  effective  use  of  specific 
and  pertinent  information  about  his  system’s 
performance  can  the  Federal  ADP  manager 
properly  fulfill  his  responsibilities.  This  in¬ 
formation  can  be  acquired  through  the  appli¬ 
cation  of  existing  measurement  and  evaluation 
technology  in  the  form  of  a  vigorous  and  effec¬ 
tive  CPM  program.  It  is  therefore  recommended 
that  a  Computer  Performance  Management 
program  be  established  at  all  Federal  ADP 
facilities. 
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Appendix:  Sample  CPM  Reports 


Turnaround  Time  Report 


%  of  Jobs 

Meeting  Guaranteed 
Turnaround  Times 


Frequency :  Weekly 

Performance  Criterion: 

1.  X%  of  all  jobs  in  each  class 
shall  satisfy  the  guaranteed 
turnaround  time  requirements 
of  that  class. 
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System  Reliability  Report 


Average  Length  of 
System  Interruptions 
(minutes) 


Maximum  Length  of 
Any  System 
Interruption 
(minutes) 


Frequency  of  System 
Interruptions 


Frequency :  Daily 

Performance  Criteria: 

1.  The  average  length  of  system 
interruptions  shall  be  less 
than  X  minutes. 

2.  No  interruption  shall  last 
longer  than  Y  minutes. 

3.  There  shall  be  no  more  than 
Z  interruptions  per  day. 
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User  Support  Report 


Number  of 
User  Inquiries 


Frequency :  Weekly 

Performance  Criterion: 

1.  The  number  of  inquiries  to  the  user 
support  group  shall  not  exceed  X 
per  week. 
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of  Documents,  Government  Printing  Office,  Washington, 
D.C.  20U02. 

Order  following  NBS  publications — NBSIR's  and  FIPS 
from  the  National  Technical  Information  Services, 
Springfield,  Va.  22161. 

Federal  Information  Processing  Standards  Publications 
(FIPS  PUBS) — Publications  in  this  series  collectively 
constitute  the  Federal  Information  Processing  Stand¬ 
ards  Register.  Register  serves  as  the  official  source  of 
information  in  the  Federal  Government  regarding  stand¬ 
ards  issued  by  NBS  pursuant  to  the  Federal  Property 
and  Administrative  Services  Act  of  1949  as  amended, 
Public  Law  89-306  (79  Stat.  1127),  and  as  implemented 
by  Executive  Order  11717  (38  FR  12315,  dated  May  11, 
1973)  and  Part  6  of  Title  15  CFR  (Code  of  Federal 
Regulations). 

NBS  Interagency  Reports  (NBSIR) — A  special  series  of 
interim  or  final  reports  on  work  performed  by  NBS  for 
outside  sponsors  (both  government  and  non-govern¬ 
ment).  In  general,  initial  distribution  is  handled  by  the 
sponsor;  public  distribution  is  by  the  National  Techni¬ 
cal  Information  Services  (Springfield,  Va.  22161)  in 
paper  copy  or  microfiche  form. 

:ription  services 

Superconducting  Devices  and  Materials.  A  literature 
survey  issued  quarterly.  Annual  subscription:  S30.00 '. 
Send  subscription  orders  and  remittances  for  the  pre¬ 
ceding  bibliographic  services  to  National  Bureau  of 
Standards,  Cryogenic  Data  Center  (275.02)  Boulder, 
Colorado  80302. 
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